WHAT IS CLAIMED IS: 

1 . A method of selecting a mode for the encoding of a macroblock in a video 
encoder, the method comprising: 

performing a motion search to select a motion vector; 

determining a residual error for the motion vector with a plurality of 
macroblock modes; 

estimating a coding cost from motion vectors for at least some of the plurality 
of macroblock modes; and 

selecting the mode for the encoding of the macroblock based on both the 
residual error and the coding cost associated with the mode. 

2. The method as defined in Claim 1, wherein the residual error is computed 
according to a sum of absolute differences (SAD) calculation (LI -norm). 

3. The method as defined in Claim 1, wherein the residual error is computed 
according to a sum of squares calculation (L2-norm). 

4. The method as defined in Claim 1, wherein the coding cost for an Inter 
macroblock with zero motion vector is zero. 

5. The method as defined in Claim 1, further comprising using different criteria 
to calculate the coding cost depending on whether the picture is a B-picture or is a P-picture. 

6. The method as defined in Claim 1, further comprising using different criteria 
to calculate the coding cost depending on whether the picture is interleaved or is progressive. 

7. A method of selecting a mode for the encoding of a macroblock (MB) in a 
video encoder, the method comprising using both an indication for residual error and a 
coding cost for a motion vector for a mode to determine which mode is selected for the 
macroblock. 

8. The method as defined in Claim 7, wherein the motion vector (MV) is a 
differential motion vector (DMV). 

9. The method as defined in Claim 7, wherein the residual error is at least 
partially computed by reusing an LI -norm calculation from a motion search. 

10. The method as defined in Claim 7, wherein a sum of absolute differences 
(SAD) calculation is used for the residual error, further comprising: 
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adjusting the SAD calculation is made based on the coding cost for the mode 
to adjust the SAD calculation; 

comparing the adjusted SAD calculation for the mode to another adjusted 
SAD calculation for another mode; and 

selecting the mode with the lower adjusted SAD calculation. 

11. A video encoder that is configured to select a mode for the encoding of a 
macroblock (MB), the video encoder comprising means for using both an indication for 
residual error and a coding cost for a motion vector for a mode to determine which mode is 
selected for the macroblock. 

12. The video encoder as defined in Claim 11, wherein the residual error is at 
least partially computed by reusing an LI -norm calculation from a motion search. 

13. A computer program embodied in a tangible medium comprising a module 
with instructions for selecting a mode for the encoding of a macroblock (MB), the computer 
program including instructions for using both an indication for residual error and a coding 
cost for a motion vector for a mode to determine which mode is selected for the macroblock. 

14. The computer program as defined in Claim 13, wherein the residual error is at 
least partially computed by reusing an LI -norm calculation from a motion search. 

15. A circuit configured to select a mode for the encoding of a macroblock in a 
rate and quality efficient manner, the circuit comprising: 

a circuit configured to perform a motion search to select a motion vector; 

a circuit configured to determine a residual error for the motion vector with a 
plurality of macroblock modes; 

a circuit configured to estimate a coding cost associated with the motion 
vectors for at least some of the plurality of macroblock modes; and 

a circuit configured to select the mode for the encoding of the macroblock 
based on both the residual error and the coding cost associated with the mode. 

16. The circuit as defined in Claim 15, wherein the residual error is computed 
according to a sum of absolute differences (SAD) calculation (LI -norm). 

17. A video encoder embodying the circuit defined in Claim 15, 
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